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INTRODUCTION 


The  goal  of  this  project  is  to  record  neural  activity  of  serotonin  (5HT)  and 
dopamine  (DA)  neurons  in  rats  during  the  performance  of  a  reinforcement 
learning  task.  We  are  interested  in  how  neural  activity  within  these  ascending 
modulatory  systems  is  temporally  related  to  events  during  learning  and  decision 
making.  Electrophysiological  studies  of  ventral  tegmental  area  (VTA)  neurons  in 
monkeys  by  Wolfram  Schultz’s  group  have  led  to  a  breakthrough  in 
understanding  of  the  DA  system  by  showing  that  their  phasic  firing  patterns 
correspond  to  an  important  variable  posited  by  reinforcement  learning  theory,  the 
reward  error  (or  temporal  difference)  signal.  These  studies  have  not  yet  been 
replicated  in  rats.  In  doing  so,  the  ongoing  work  will  provide  a  window  onto  the 
function  of  the  DA  system  that  is  complementary  to  the  rich  existing  knowledge 
from  pharmacological  and  biochemical  assays.  Even  less  is  known  about  the 
relationship  between  dorsal  raphe  (DR)  5HT  neuron  firing  and  decision  variables 
in  any  species.  Indeed,  there  are  no  studies  characterizing  the  behavior  of  5HT 
neurons  during  the  performance  of  a  behavioral  task  in  any  species.  The  major 
goal  of  the  ongoing  work  is  to  identify  phasic  behavioral  correlates  of  5HT  neural 
activity.  Over  the  course  of  this  project,  we  have  overcome  many  of  the  technical 
hurdles  associated  with  recording  from  raphe  neurons  in  awake  behaving 
animals.  We  have  found  that  raphe  neurons  show  phasic  modulation  of  firing 
rates  to  the  order  of  milliseconds  in  response  to  specific  task  events.  Thus,  we 
have  made  significant  strides  towards  our  goal  of  characterizing  neural  activity 
within  the  serotonin  and  dopamine  systems  during  goal  directed  behavior. 

RESULTS 


Anesthetized  Experiments 

Localizing  and  Identifying  Single  Cells  within  the  Dorsal  Raphe  Nucleus 


Forebrain  projecting  serotonin  neurons  are  principally  located  within  two 
midline  nuclei  -  the  dorsal  raphe  and  median  raphe  nuclei.  Because  these  nuclei 


Figure  1:  Neuronal  cell  types  from  dorsal  raphe  nucleus  of  anesthetized  rats. 


Left  panel :  wide  action  potential  waveform  of  a  putative  5HT  neuron  and  spike  train 
exhibiting  its  regular  firing  pattern.  Right  panel :  narrow  action  potential  waveform  of  a 
fast  firing  non-5HT  neuron  and  an  example  trace  showing  bursty,  irregular  firing 
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contain  a  variety  of  cell  types  in  addition  to  serotonin  neurons,  it  was  important  to 
establish  our  ability  to  reliably  identify  distinct  cell  classes  within  the  raphe 
nucleus  using  standard  electrophysiological  criteria  prior  to  the  initiation  of  awake 
recordings.  Serotonin  neurons  are  identified  by  their  wide  action  potential,  low 
spontaneous  firing  rate  and  regular  firing  pattern.  In  addition  serotonin  neurons 
are  inhibited  by  the  5HT1a  receptor  agonist.  We  carried  out  a  series  of 
recordings  using  sharp-tungsten  electrodes  in  the  anesthetized  rat.  Consistent 
with  previous  studies,  we  find  at  least  two  distinct  cell  types,  examples  of  which 
are  shown  in  Figure  1.  The  first  cell  class  consists  of  putative  GABAergic 
interneurons  characterized  by  their  narrow  waveform  and  high  firing  rate  (Fig.1 
left  panel).  The  second  cell  class  consisting  of  cells  with  wide  action  potentials 
and  spontaneous  firing  rates  of  around  0.5-2  Hz  is  characteristic  of  serotonin 
producing  neurons  (Fig.  1,  right  panel).  Pharmacological  experiments  are 
underway,  however  histological  localization  of  our  electrodes  and  other  firing 
characteristics  provide  evidence  that  these  are  putative  serotonin  neurons. 

Awake  Experiments 

Technical  hurdles  and  solutions  for  raphe  recordings  in  awake  rats 

Since  we  are  especially  interested  in  characterizing  interactions  between 
cells  within  the  raphe,  we  decided  to  use  tetrode  (four  channel)  recording 
electrodes  in  favor  of  single  wires  for  awake  experiments.  By  using  a  process 


Figure  2:  Novel  approach  for  raphe  recordings  in  awake  rats  using  tetrodes. 


Left  panel:  Schematic  diagram  of  guide  cannula  and  tetrode  drive  implantation  into  dorsal 
raphe  nucleus.  Right  panel:  Dil  track  created  by  tetrode  penetration  visualized  by  fluorescence 
microscopy  clearly  shows  track  passing  through  dorsal  raphe  (DR).  Scale  bar:  0.5  mm 

analogous  to  triangulation,  tetrodes  are  capable  of  segregating  signal  sources 
(i.e.  individual  neurons)  thereby  allowing  simultaneous  recording  of  several 
neurons.  However,  in  the  process  of  switching  from  tungsten  electrodes  to 
tetrodes  a  number  of  technical  hurdles  had  to  be  overcome.  In  particular,  the 
anatomical  location  of  the  raphe  nuclei  is  such  that  the  most  direct  electrode 
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penetration  passes  thru  both  the  midline  sinus  (plexus  of  blood  vessels)  and  the 
cerebral  aqueduct.  While  sharp  electrodes  are  able  to  puncture  the  sinus  and  the 
thick  dura  below  it,  tetrodes  are  not  stiff  enough  to  pass  through.  While  it  is 
possible  to  bypass  the  sinus  by  penetrating  laterally  at  an  angle,  it  is  very  difficult 
for  reliable  stereotactic  targeting  since  the  raphe  nucleus  is  a  small  structure.  It 
also  limits  our  ability  to  record  from  both  the  dorsal  and  medial  raphe  nucleus  on 
a  single  electrode  penetration.  As  a  consequence,  we  developed  a  novel  two- 
step  approach  for  tetrode  targeting  and  implantation.  In  the  first  part  of  the 
procedure,  a  hole  is  drilled  in  the  skull  on  the  midline  above  the  raphe  (-7.5  mm 
A.P.  relative  to  bregma)  and  a  stainless  steel  guide  cannula  (18.5  gauge)  with  a 
stylet  to  maintain  patency  is  lowered  into  the  brain.  The  cannula  tip  is  at  a  depth 
of  ~  3.0  -  3.5  mm  from  the  skull  surface  (~  2mm  above  the  raphe).  Tetrodes  are 
lowered  into  the  brain  through  the  cannula.  Using  this  procedure  we  are  able  to 
stereotactically  localize  tetrodes  into  the  raphe  nucleus  as  demonstrated  by 
fluorescent  labeling  of  tetrode  tracks  stained  with  Dil  (Fig.  2). 

Raphe  Recordings  in  the  Awake  Rat:  Neuronal  cell  types 

Using  the  above  mentioned  method  we  successfully  recorded  neural 
activity  from  DR  nucleus  of  awake  freely  behaving  rats.  Chronically  implanted 


Figure  3:  Neuronal  cell  types  from  DR  nucleus  of  awake  rats 
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Narrow  spiking,  high  firing  rate,  non-serotonergic  neuron 
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Firing  properties  of  putative  5HT  neuron  (A)  and  non-5HT  neuron  (B) 
For  each  neuronal  cell  type,  typical  action  potential  waveform  (left, 
red  trace),  Log  isi  distribution  (right,  blue  trace)  and  representative 
spike  train  (bottom)  are  shown.  Scale  bar:  2  sec 


animals  recovered  completely  from  surgery,  were  healthy  and  showed  no  gross 
behavioral  deficits.  Their  performance  on  2  odor  discrimination  task  was  identical 
to  that  before  surgery.  Putative  5HT  neurons  were  identified  on  the  basis  of  their 
wide  action  potential  and  slow  regular  firing  pattern  (Fig.  3A).  Non-5HT  neurons 
had  a  narrow  spike  and  showed  variation  in  their  firing  rates.  Fig.  3B  shows  an 
example  of  a  narrow  spiking  non-5HT  neuron  with  high  spontaneous  firing  rate 
and  irregular  firing  pattern.  A  small  subset  of  narrow  spiking  neurons  also  had  a 
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slow  spontaneous  firing  rate.  Thus,  we  are  able  to  record  from  raphe  neurons  in 
awake  rats  using  tetrodes. 

Tonic  changes  in  firing  rate  across  sleep  wake  cycle 

Previous  studies  in  cats  and  rats  have  demonstrated  firing  rate  changes  in 


Figure  4:  Tonic  changes  in  firing  rate  of  putative  5HT  neuron  across  sleep  wake  cycle 


Red  traces  are  hippocampal  EEG  recordings  and  black  ticks  are  spikes  aligned  to  EEG  trace 
during  two  epochs  showing  marked  inhibition  of  firing  during  large  amplitude  activity 
associated  with  slow  wave  sleep  (SWS)  and  increased  firing  at  transitions  from  SWS  to  wake. 
Scale  bars:  left:  Imin  and  right:  5  sec 


serotonin  neurons  across  the  sleep  -  wake  cycle.  In  our  recordings  we  have  also 
found  neurons  that  exhibit  these  properties.  An  example  neuron  is  shown  in  Fig. 
4.  This  is  the  same  neuron  shown  in  Fig.  3A. 

This  neuron  shows  a  marked  suppression  in  firing  as  the  animal  transitions  from 
waking  to  slow  wave  sleep  (SWS).  SWS  is  characterized  by  large  amplitude 
activity  in  the  1-5  Hz  in  the  hippocampal  EEG  [1].  While  the  tonic  firing  rate  does 
not  change  much  over  the  course  of  a  behavioral  epoch,  the  neuron  increases  its 
firing  rate  transiently  at  the  beginning  of  a  sleep  to  waking  transition.  Firing  rate 
changes  with  behavioral  state  have  been  reported  for  serotonin  neurons  [2], 
however,  there  is  no  evidence  in  literature  for  increased  firing  at  state  transitions. 


Raphe  recordings  during  goal  directed  behavior 

Behavioral  correlates  of  raphe  neuronal  firing  -  Phasic  modulation 
Extensive  pharmacological  and  lesion  studies  suggest  a  role  for  forebrain 
serotonin  in  reward  learning  as  well  as  impulsivity  [3].  However,  there  are  no 
studies  of  raphe  neuronal  responses  during  behavior.  Thus  our  understanding  of 
serotonin  function  in  the  brain  is  limited  to  long  term  effects  on  animal  behavior 
due  to  inherent  large  time  constants  of  action  for  drugs  and  lesions  (several 
minutes  to  days).  Raphe  recordings  from  awake  animals  also  focus  on 
elucidating  tonic  firing  rate  changes.  Thus  there  is  no  data  available  on  response 
properties  of  raphe  neurons  during  goal  directed  behavior.  This  approach  has 
been  especially  successful  in  the  study  of  another  neuromodulatory  system 
namely  dopamine  [4].  Therefore  one  of  the  major  goals  of  this  project  was  to 
record  from  raphe  neurons  during  goal  directed  behavior  and  correlate  phasic 
modulation  of  firing  rate  with  specific  behavioral  events  over  short  timescales  of 
milliseconds.  While  serotonin  neurons  are  responsible  for  5HT  release  in  target 
structures,  their  firing  and  therefore  release  is  regulated  by  non-5HT  neurons  in 
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raphe,  (e.g.  GABAergic  interneurons).  Therefore  we  were  interested  in  recording 
from  both  5HT  and  non  5HT  neurons.  Over  the  past  several  months  we  have 
obtained  several  recordings  from  neurons  in  the  DR  while  the  rat  is  engaged  in  a 
2  odor  discrimination  task. 

Water  deprived  rats  were  trained  to  poke  into  a  center  port  where  on  a  given  trial 
one  of  two  odors  was  delivered.  Each  odor  was  associated  with  availability  of 
water  at  one  of  two  choice  ports  (Left  and  Right).  Depending  upon  odor  identity 
the  rat  learnt  to  choose  the  correct  port  to  receive  water  reward.  Rats  performed 
this  task  very  well  with  more  than  90%  correct  responses[5].  There  was  no 
change  in  performance  after  drive  implantation  surgery.  Timing  of  all  task  events 
was  recorded  with  millisecond  precision  enabling  us  to  align  spikes  to  various 
behavioral  events  and  investigate  phasic  neuronal  responses.  With  this  task  we 
can  study  correlations  in  firing  of  raphe  neurons  during  various  aspects  of 
behavior  including  sensory,  motor,  decision  and  reward  processes. 

Modulation  during  odor  sampling: 

Raphe  neurons  heavily  project  to  the  MOB  as  well  as  other  olfactory  areas. 
Activity  of  subset  of  serotonin  neurons  increases  during  sniffing  [6].  Similarly, 
Barry  Jacobs  [7]  postulated  the  motor  hypothesis  outlining  a  model  in  which 
activation  of  serotonin  neurons  facilitates  motor  output  while  simultaneously 
inhibiting  sensory  processing.  Thus  we  expected  to  see  modulation  of  neuronal 
firing  during  odor  sampling  phase  of  the  task. 

As  expected,  33%  of  neurons  recorded  showed  odor  sampling  related 
modulation.  There  were  examples  of  both  activation  and  inhibition  (Fig.  5).  Top 


Figure  5:  Firing  rate  modulation  during  odor  sampling. 


Top  panel:  Peri-stimulus  Time  Histogram  (PSTH)  of  putative  5HT  neuron  showing 
inhibition  during  odor  sampling.  Spikes  are  locked  to  time  of  Odor  onset  and  averaged 
over  trials.  Bottom  panel:  PSTH  of  a  non-5HT  neuron  showing  activation  to  1  odor  from 
the  pair.  Red  and  blue  traces  denote  trials  with  the  2  odors. 


trace  is  of  a  putative  5HT  neuron  same  as  in  Fig.  3a  and  4.  This  neuron  was 
inhibited  during  odor  sampling  phase  consistent  with  prediction  of  the  motor 
hypothesis.  However,  neuron  in  the  bottom  trace  (a  narrow  spiking,  high  firing 
rate,  non-5HT  neuron)  showed  odor  specific  activation  only  to  1  odor  from  the 
pair.  While  this  neuron  could  be  a  non-5HT  GABAergic  interneuron  that  might 
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inhibit  the  5HT  neuron,  the  stimulus  specificity  of  the  activation  is  hard  to  explain. 
This  specificity  may  be  imparted  by  other  higher  order  differences  between  the  2 
stimulus  reward  contingencies. 

Modulation  during  movement: 

Serotonin  has  been  hypothesized  to  play  an  important  role  in  central  control  of 
motor  function.  Raphe  neurons  change  their  firing  rate  during  treadmill  induced 
locomotion  as  well  as  other  motor  activities  such  as  grooming  and  other  central 
pattern  generator  (CPG)  mediated  behaviors  [6].  Consistent  with  this 
expectation,  Figure  6  shows  a  neuron  that  increased  its  firing  rate  during 
movement  from  the  center  port  to  the  water  port  on  both  sides.  However,  this 
increase  was  not  observed  while  approaching  odor  port  or  going  away  from 
water  port.  This  leads  us  to  speculate  that  this  neuron  might  be  modulating  a 
stereotyped  motor  pattern  since  all  other  movements  are  not  as  stereotyped  as 
movement  from  center  port  to  water  port(s).  However,  the  specificity  may  even 
be  correlated  to  reward  expectation.  These  possibilities  can  be  teased  out  by 
comparing  trials  on  which  animal  expected  a  reward  to  trials  where  the  animal 
did  the  same  movement  but  did  not  expect  a  reward  by  employing  a  fixed  ratio 


Figure  6:  Modulation  of  raphe  firing  during  movement. 


Movement  to  water  port 

- ► 


Ttoe 

Top  panel  shows  a  raster  plot  of  a  non-5HT  neuron  locked  to  exit  from  the  odor  port.  Bottom 
panel  shows  PSTH  of  the  same  neuron.  Blue  and  red  traces  are  for  movement  to  left  and  right 
sides  respectively.  Firing  rate  increases  as  the  animal  approaches  the  water  port.  Entry  into 
water  port  is  denoted  by  blue  dots.  Trials  are  sorted  by  increasing  movement  time. 


reward  schedule.  We  would  also  like  to  test  the  effect  of  increased  effort  on  such 
responses  since  serotonin  has  been  implicated  in  such  behaviors  [8]. 

Activity  during  reward  delay: 

Serotonin  is  known  to  be  involved  in  behaviors  related  to  time  discounting  such 
as  inter-temporal  choice  behavior,  temporal  discounting,  delayed  reinforcement 
as  well  as  impulsive  choice  behavior  through  pharmacological  and  lesion  studies 
[9].  However  how  this  relates  to  changes  in  firing  rate  of  raphe  neurons  is  not 
known.  We  expect  raphe  neuronal  activity  to  be  strongly  influenced  by  delay  to 
reward.  To  study  this  aspect  of  serotonin  function,  in  our  task  there  was  a  fixed 
delay  between  water  port  poke  and  reward  delivery.  A  neuron  that  modulates 
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firing  rate  during  delay  period  is  shown  in  Fig.  7.  This  neuron  monotonically 


Figure  7:  Firing  rate  modulation  during  reward  delay. 
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Top  panel  shows  a  raster  plot  of  a  non-5HT  neuron  locked  to  entry  into  the  water  port. 

Bottom  panel  shows  PSTH  of  the  same  neuron.  Red  trace  is  the  rewarded  responses 
while  blue  trace  are  randomly  interleaved,  unrewarded  correct  responses.  Note  monotonic 
decrease  in  firing  rate  during  reward  delay.  Blue  dots  in  top  trace  denote  time  of  reward 
delivery,  red  dots  denote  exit  from  water  port. 

decreased  its  firing  rate  until  the  expected  time  of  reward  and  stayed  inhibited 
during  the  time  of  reward  delivery.  However,  it  showed  an  anticipatory  increase  in 
firing  rate  before  the  water  valve  was  turned  off  (again  a  fixed  period  of  0.5  s). 
Thus  the  neuron  was  inhibited  at  time  of  expected  reward.  Interestingly  in  case  of 
randomly  unrewarded  correct  trials,  the  neuron  continued  to  be  inhibited  until  the 
rat  poked  its  nose  out  of  the  water  port.  Such  delay  related  activity  during  reward 
period  may  be  involved  in  temporal  discounting  computations.  These  responses 
can  be  further  tested  by  changing  distribution  of  reward  delays. 

Phasic  activation  by  water  valve  sound: 

It  was  somewhat  surprising  that  50%  neurons  showed  phasic  activation  to  sound 
Figure  8:  Phasic  activation  of  putative  5HT  neuron  by  sound  of  water  valve. 
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Top  panel  shows  a  raster  plot  of  a  5HT  neuron  locked  to  water  valve  onset.  Bottom  panel 
shows  PSTH  of  the  same  neuron.  Red  and  blue  traces  denote  trials  with  the  2  odors.  The 
neuron  shows  a  sharp  peak  in  activity  immediately  after  water  valve  onset  resulting  in  a  loud 
click. 

of  opening  water  valve  (Fig.  8).  Latency  of  activation  was  ~  20  ms.  There  have 
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been  earlier  reports  of  tone  responsive  raphe  neurons  [10].  There  can  be  various 
explanations  for  these  sound  evoked  responses.  Raphe  nucleus  sends 
projections  to  all  areas  along  the  auditory  pathway  from  cochlear  nucleus  to 
primary  auditory  cortex.  Thus  the  phasic  response  could  be  purely  sensory 
involved  in  auditory  processing  and  thus  conform  to  the  motor  hypothesis 
discussed  earlier.  Dorsal  raphe  projects  heavily  to  the  inferior  colliculus  (1C), 
especially  to  those  layers  that  integrate  auditory  and  non-auditory  inputs  like  from 
the  amygdala  [11].  Thus,  serotonin  may  modulate  responses  of  1C  neurons  to 
these  non-auditory  inputs  that  may  be  involved  in  manifestation  of  defensive 
behaviors  in  response  to  aversive  stimuli.  This  goes  hand  in  hand  with  the 
proposed  role  of  serotonin  in  anxiety  and  stress  related  behaviors.  Thus,  it  will  be 
interesting  to  see  if  activitation  is  purely  auditory  or  if  it  is  also  modulated  by  other 
factors  like  salience,  aversive  conditioning  or  reward. 

KEY  RESEARCH  ACCOMPLISHMENTS 

•  Developed  and  characterized  rodent  behavior  in  a  novel  two-alternative 
olfactory  discrimination  task  and  successfully  adapted  it  to  mice. 

•  Developed  novel  cannulated  microdrive  assembly  for  long  term  tetrode 
recordings  in  deep  brainstem  structures. 

•  Recorded  neural  activity  of  raphe  neurons  during  goal  directed  behavior 
and  found  phasic  modulation  of  firing  rates  correlated  to  specific 
behavioral  events. 

REPORTABLE  OUTCOMES 

1.  Work  toward  the  degree  of  Ph.D.  for  Sachin  Ranade,  Neurobiology  and 
Behavior  Program,  Stony  Brook  University. 

2.  Obtained  research  support  from  the  Thomas  Hartman  Foundation  For 
Parkinson's  Research. 

3.  Presented  poster  at  Computational  and  Systems  Neuroscience  2006 
(CoSyNe  2006)  in  Salt  Lake  City,  Utah. 

CONCLUSIONS 


The  goal  of  this  project  was  to  record  activity  of  serotonin  and  dopamine 
neurons  during  goal  directed  behavior.  During  the  course  of  the  project  we  have 
overcome  a  number  of  technical  hurdles  in  order  to  successfully  record  neural 
activity  of  serotonergic  and  non-serotonergic  neurons  within  the  midbrain  raphe 
nuclei  of  behaving  rats.  We  found  that  contrary  to  earlier  studies  that  reported 
mainly  tonic  long  timescale  changes  in  firing  rates,  raphe  neurons  phasically 
respond  to  task  events  on  a  short  timescale  of  several  milliseconds  during 
various  phases  of  the  task  spanning  sensory,  motor  and  reward  related 
processes.  We  are  encouraged  by  these  results  and  are  confident  that  this 
approach  of  recording  from  raphe  neurons  during  goal  directed  behaviors  will 
provide  a  new  understanding  of  function  of  serotonin  in  aspects  of  cognition, 
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emotion  and  behavior.  We  now  plan  to  investigate  raphe  neuronal  responses 
during  specific  behaviors  such  as  delayed  reinforcement  and  reversal  learning 
which  are  perturbed  by  serotonin  dysfunction.  We  also  plan  to  study  the 
interaction  between  raphe  neuronal  firing  and  sniffing.  Having  established  the 
technically  challenging  raphe  recording  technique  we  can  now  focus  our  attention 
on  performing  dual  serotonin  and  dopamine  neuron  recordings  in  the  context  of 
reinforcement  learning  to  fulfill  the  other  goal  of  this  project.  Finally  we  want  to 
complement  our  recording  techniques  with  the  immense  potential  afforded  by 
mouse  transgenic  and  knockout  technologies  and  targeted  gene  delivery 
methods  in  order  to  move  from  correlations  with  behavior  to  a  more  mechanistic 
understanding  by  manipulating  behavior. 
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